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Detecting a content item in a digital video stream 



in 



The invention relates to a method of detecting a boundary of a content item 
a digital video stream. The invention also relates to a device for detecting a boundary of a 
content item in a digital video stream. The invention former relates to a receiver for receiving 
at least one content item in a digital broadcast video stream, and a video recorder for 
recording at least one TV program. 



US patent application US2002/0 186768 describes a method of content 
detection based on data obtained in a process of compressing video, e.g. by a MPEG encoder. 
The MPEG encoder encodes video data from a TV broadcaster, the Internet, etc. The MPEG 
encoder generates the data such as a current bit rate value combined with a quantizer value to 
indicate fast varying or very complex scenes in the video, e.g. to indicate visual complex 
pictures or pictures containing fast motion. This encoding bit rate measures an amount of bite 
required for encoding a piece of video, and it usually increases for fast moving pictures and 
visually complex scenes. 

However, the encoding bit rate can not reliably indicate, for example, whether 
the video comes from a movie containing fast motion or a commercial with fast moving 
scenes. The known method can not reliably indicate a boundary of a content item using the 
encoding bit rate. Moreover, foe method known from US2002/0 186768 can be used only 
when the MPEG encoder is available. 



It is an object of foe present invention to obviate foe drawbacks of foe known 
method, and to provide a method of detecting foe boundary of a content item in a digital 
video stream which is more reliable and can be used without any MPEG encoder. 

This object is realized in that foe method of foe present invention comprises 

the steps of: 

deteimining an average bit-rate of foe video stream over a period of time, and 
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detecting a change of the average hit-rate indicating the boundary of the 

content item. 

The content items may vary in their quality and, hence, hit rates. For example, 
commercials may have higher hit rate because companies usually pay for high-quality 
5 advertisement material. In case of a broadcast digital video stream, broadcasters may, for 
example, keep the hit rate of a channel constant hut allocate different hit rates to content 
items according to a type of content 

The hit-rate is a rate of data allocated to a content item in the digital video 
stream may he determined. The hit-rate may be indicated as additional information in the 
10 digital stream. For example, in the digital video broadcasting (DVB) a number of streams 
carrying video, audio, control data formed into packets of a certain type may be transmitted. 
With the video data in the packets having a predetermined or indicated size, it is possible to 
determine the bit rate of the video stream. 

The average bit rate may be calculated in various ways, e.g. simply for 
15 successive periods of time or, alternatively, a moving average of the bit rate may be 

determined. Of course, other manners of calculating a value of the bit rate over a period of 

time may be envisaged. 

The boundary of the content item may be ascertained based on detecting the 
change of the average bit rate, for example, by detennining a deviation of the average bit rate 
20 value exceeding a predetermined threshold, a deviation of the change of the average bit rate 
value exceeding a certain percentage of said value, etc. 

The present invention has an advantage that the average bit rate is more 
reliable and robust than the encoding bit rate known in prior art. The determination of the 
average bit rate over the period of time smoothes variations which are not indicating real 
25 changes in the type of content 

A further problem associated with hie known method of content detection is 
found when foe content is in an encrypted digital stream. The method known in the prior art 
document US2002/0186768 can not be applied to the encrypted digital stream. This known 
method would require decrypting the stream, decoding it to uncompress the stream and re- 
30 encoding foe stream to compute the encoding bit stream. The known method can be applied 
only to MPEG data representing groups of pictures in the stream, and it is based on a mean 
absolute difference (MAD) value. This MAD value represents an average of magnitudes of 
vectors used to describe movement of video blocks in frames. Obviously, this information is 
not accessible in the encoded video stream. 



PHNL030869EPP 



3 18.07.2003 
In turn, the method of detecting the boundary of the content item according to 
the present invention can be applied to the encrypted digital video stream. The method is 
based on the average bit rate of the stream and it is not affected substantially in case of the 
encrypted stream. 

The method of the present invention has an advantage that it works fast for the 
encrypted streams, i.e. the average bit rate can be easily calculated also for the encrypted 
streams without any decryption. Moreover, if the boundary of the content item in the 
encrypted stream is detected, only a part of the digital encrypted stream belonging to the 
content item may need to be decrypted for further purposes such as recording the content 
item, the rest of the stream need not be decrypted. 

In one of the embodiments, the method of the present invention can be applied 
in an Electronic Program Guide (EPG) of the TV system. In the EPG, broadcast schedule 
data (EPG data) indicating a beginning and/or end of broadcasting at least one content item 
may be available in advance, i.e. before the broadcast of the content item. According to the 
method of the present invention, the detection of the boundary of the content item may be 
used for verifying whether the EPG data are in accordance with the detected boundary of a 
respective content item in the video stream. This may be utilized in a video recorder for 
precisely controlling the beginning and end of recording the broadcast content item. 

The object of the present invention is also realized in that the invention, 
provides for the device for detecting a boundary of a content item in a digital video stream, 
the device comprising: 

means for determining an average bit-rate of the video stream over a period of 

time, 

means for detecting a change of the average bit-rate indicating the boundary of 
the content item. 

Said device may be incorporated into a receiver for receiving at least one 
content item in a digital broadcast video stream; or a video recorder for recording at least one 
TV program, comprising such a receiver and means for recording the TV program based on 
its detected boundary in the video stream. 

These and other aspects of the invention will be further elucidated and 
described with reference to the accompanying drawings, wherein: 

Fig. 1 shows an embodiment of the method of the present invention; 
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Fig. 2 depicts a diagram showing an example of the calculated average bit rate 

of Ae video stream; 

Fig. 3 depicts an enlarged diagram showing an example of the calculated 

average bit rate at the end of the content item; 
S Fig. 4 shows a diagram illustrating operation of die device suitable for 

implementing the present invention* 

Fig. 1 shows an embodiment of a method of detecting a boundary of a content 
1 0 item in a digital video stream. The video stream may be obtained by reading out information 
from a data carrier such as a CD-ROM disk (Compact Disk Read Only Memory), a DVD 
disk (Digital Versatile Disk), a magnetic carrier, etc. Alternatively, the video data may be 
received in a known manner from a video broadcaster, e.g. using digital video broadcasting 
(DVB), video on demand systems, the Internet, etc. 
15 The bit rate of the video stream may be determined at step 110. The video 

stream is usually packetized and kept in so-called frames. The frames may have a header and 
a body, and the header comprises information about the data contained in the body. The 
header may indicate information related to a bit rate usually expressed as the number of bits 
transmitted per second. For example, the digital stream, according to the MPEG standard, 
20 obtained using the DVB broadcasting or from the DVD disk track may contain a plurality of 
the packetized elementary streams. The packets of the elementary stream maybe of a fixed or 
variable size. Hie packets may have different types and belong to different elementary 
streams. Therefore, packets belonging to elementary streams carrying pay load information, 
e.g. so-called transport streams, may need to be filtered out The fixed-size packet of the 
25 transport stream may comprise, for example, 184B of payload and 4B of the header. 

In another example, the video stream may be obtained in a Digital Video (DV) 
standard. For instance, DV packets may be obtained from a DV equipment, e.g. a DV 
camcoder or a DV recorder, via EBEE1394 protocol (commonly known as FireWire). In the 
DV format, a video frame according to the NTSC (National Television System Committee) 
30 video method (525 lines/frame, 30 video frames per second) consists of 1500 DV blocks that 
have the size of 80B. The DV/IEEE1394 packets obtained from the DV equipment may have 
a format with a header of 12B and a body of 480B (6 DV blocks). Thus, the video frame may 
comprise 250 DV blocks. Having this information, it is possible to determine the bit-rate of 
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the video stream obtained from such DV equipment The bit rate of the video stream in other 
formats may be computed. 

At step 120, a period of time is determined for calculating an average bit rate 
of the video stream. The period may be pre-set to a value of several seconds, two-five 
seconds or more (step 121). Alternatively, the period of time may be variable. For example, 
shot cuts, e.g. scene changes, in the video content item may be detected at step 125 and the 
period of time may be adjusted to the detected shot-cuts accordingly at step 126, so that a 
particular period of time may be equal to a period between scene changes. In a farther 
example, if the time period between detected scene changes is too large, e.g. exceeding some 
pre-set maximum threshold, it may be split into shorter time periods over which the average 
bit rate may be calculated. The changes of scene may be determined, for example, using a 
scene change detector known from a document EP 1,006,685. The scene change detector 
calculates an inverse correlation value of successive frame images. The correlation value is 
estimated based on absolute values of differences between pixel values of corresponding 
pixels of the images. The scene change is determined when the correlation value exceeds a 
predetermined threshold value. Fast motion detectors known in the prior art may also be used 
for detecting scene changes. When the video stream is encoded, partial decoding of the 
stream to obtain distribution of the headers of the packets and detect the scene changes may 
be sufficient, without complete decoding of the video stream, e.g. the MPEG stream. 

At next step 130, the average bit rate of the video stream over the determined 
period of time may be estimated in different manners. The average bit rate may be computed 
by adding values of the bit rate over the determined period of time and dividing the sum by 
the number of values. In another example, a moving average bit rate may be calculated, e.g. 
mean value calculated over a rolling previous period of fixed length. The average of the bit 
rate may smooth small variations of the bit rate that are, e.g., due to noise instead of real 
boundary of the content item, and real changes in the type of content. 

By detecting a change of the determined average bit rate, a boundary of the 
content item may be found at step 140. For example, a threshold bit rate value may be 
compared with the determined average bit rate, and the boundary of the content item may be 
determined if the threshold value is exceeded. The threshold value may be fixed or variable. 
For example, the threshold value may be a percentage of the average bit rate within which it 
may change with respect to the values of the average bit rate calculated for the previous 
period or several preceding periods. In this case, the absolute value of the threshold for 
average bit rate is not fixed. In another example, a first derivative of the values of the average 
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bit rate for consecutive periods of time may be estimated to detect the rapid change of the 
average bit rate. 

When foe period of time for calculating foe average bit rate is not short enough 
for determining foe boundary of foe content item with acceptable accuracy, e.g. half of a 
5 second, one second, etc, a position of foe detected boundary of foe content item within foe 
corresponding period of time may be determined at step 150. The position of the boundary 
may be determined by comparing foe values of foe bit rate for progressive frames in foe video 
stream; by partitioning foe given period of time into small segments and comparing foe 
average bit rate values for each successive segment with foe period of time in which foe 
10 boundary of the content item is determined, or in other manners. 

In Fig. 2, foe average bit rate of foe video stream calculated for foe TV 
program "Presumed Innocent" received from digital broadcast is shown. The duration of foe 
program is approximately 2 hours and 55 minutes. The begin and end time of broadcasting 
foe program were based on a TV program guide (BPG) and extended for a few minutes. The 
15 real beginning and end of foe program and commercial blocks are highlighted. The vertical 
axis in Figs. 2 and 3 reports foe average bit rate in kbits per second and foe horizontal axis 
reports foe progressive frame number. 

In this example, foe average bit rate shown in Fig. 2 changes dramatically 
during program boundaries and commercial blocks. The average bit rate changed 
20 considerably in the beginning of foe program, especially in the first 1 500 frames. Three 

commercials in the program also caused a lot of changes in foe average bit rate. Finally, foe 
average bit rate changed significantly at foe end of foe program as it is shown in Figure 3. 
Figure 3 shows changes of foe average bit rate within a thousand of video frames at foe end 
of foe program. These changes may be caused by splicers used by providers of foe video 
25 content to insert digital content such as commercials into foe video content which might had 
already been digitized, without foe need for expensive decoding and re-encoding that can 
reduce foe quality of content 

Typically, foe TV broadcasters or other providers of content insert foe 
commercial blocks in different content items in a similar way, for example, in foe beginning 
30 and/or end, and may be several times in foe middle of foe content items. This pattern of a 
particular typical positioning of foe commercials within foe content items may be learnt by a 
device for detecting foe boundary of foe content item, and used for reliably detennining 
whether the detected change of foe average bit rate indicates foe commercial block of foe 
content item. Moreover, known methods of recognizing commercial blocks based on 



PHNL030869EPP 



7 18.07.2003 
analyzing audio and/or video features of the content item may be used for verifying whether 
the detected boundary belongs to the commercial. 

The method of detecting the boundary of the content item as described above 
can also be applied to encrypted video stream with content items. Most digital video streams, 
e.g. PayTV and Pay per View, are packetized in the conventional manner but the packets are 
encrypted using specific keys and methods. Conditional access systems adopted by 
broadcasters and set-top box manufacturers do not allow performing any operation on the 
stream but decoding in real time for viewing. The handling of encrypted streams will become 
very relevant for all products that have storage. 

The method according to the present invention can be applied to the encrypted 
streams because it is based on the average bit rate over the particular period of time and the 
encryption of the stream does not affect substantially the average bit rate. There is no need 
for decrypting the video stream to detect the boundary of the content item. If necessary, the 
encrypted video stream may be decrypted only in its parts where the boundary has been 
detected. This is more efficient and less data-processing intensive than decrypting or 
decoding the whole stream to detect the boundary of the content item. 

The average bit rate of the encrypted stream may be calculated, for example, 
by extracting the payload of the stream from the headers of the packets. For example, if the 
DVB stream is scrambled, foe headers may be not encoded, as it is described for scrambling 
and conditional access systems in the book "Digital Television: MPEG-1, MPEG-2 and 
principles of the DVB system", by H. Benoit, 1997, Arnold (chapter 5, pages 75-80). 

With reference to Figure 4, an embodiment of a device for detecting the 
boundary of the content item in the digital video stream is shown. The device may comprise a 
processor 410 arranged for processing the video stream and determining an average bit rate 
of foe video stream over foe period of time, and a detector 420 for detecting foe change of foe 
average bit rate indicating foe boundary of foe content item. 

The processor 410 may be arranged to perform steps 1 10 to 130 of foe method 
of foe present invention as described above. The processor may be implemented by the 
person skilled in foe art in foe known manner, e.g. the processor can be provided with a 
memory for storing instructions to be executed by foe processor for enabling it to function 
correspondingly. The detector 420 may be arranged to carry out operations disclosed above 
with reference to step 140 of foe present method. The detector may be implemented using foe 
same processor 410 or, for example, foe detector may be a separate circuit arranged for 
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comparing the average bit rate determined by the processor with the threshold bit rate value, 
and generating a signal indicating that the boundary of the content item has been detected. 

The processor 410 may be coupled to a receiver 430 for receiving at least one 
content item in the digital broadcast video stream, e.g. TV tuner, DVB-T or DVB-S receiver. 
5 Such a tuner/receiver may be conventional and implemented as well known in the art 

The device may be coupled to a recorder 440 for recording the TV program 
based on the detected boundary of the program in the stream. The recorder may be arranged 
to record the TV program based on broadcast schedule data, e.g. the EPG data indicating the 
beginning and end of the program. However, it may happen that the program is received by 
10 the receiver not according the schedule and the recorder will record not required content 
This may be obviated by means of the device for detecting the boundary of the content item 
as described above. The device may be arranged for verifying whether the broadcast schedule 
data are in accordance with the detected boundary of a respective content item. If there is 
some difference in time between said detected boundary and the schedule data, i.e. the 
15 content item is actually received later or earlier than the schedule data indicate, the recorder 
may be instructed to start recording the content item at the position where the boundary is 
determined. Furthermore, if the commercial blocks are detected in die stream by the device, 
the recorder may be instructed to skip them, and only the actual content item will be 
recorded. 

20 The various program products may implement die functions of the device and 

method of the present invention and may be combined in several ways with the hardware or 
located in different other devices. Variations and modifications of the described embodiment 
are possible within the scope of the inventive concept. Thus, for ©sample, the use of the verb 
'to comprise' and its conjugations does not exclude the presence of elements or steps other 

25 than those defined in a claim. The invention can be implemented by means of hardware 

comprising several distinct elements, and by means of a suitably programmed computer, hi 
the device claim enumerating several means, several of these means can be embodied by one 
and the same item of hardware. 
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CLAIMS: 



1- A method of detecting a boundary of a content item in a digital video stream, 

the method comprising the steps of: 

determining (130) an average bit-rate of the video stream over a period of 

time, and 

5 - detecting (140) a change of the average bit-rate indicating the boundary of the 

content item. 

2. The method of claim 1, wherein the content item is in a digital broadcast video 
stream. 

10 

3. The method of claim 1, comprising the steps of: 
detecting (125) shot-cuts in the content item, and 
adjusting (126) the period of time to the detected shot-cuts. 

1 5 4. The method of claim 1 , wherein a moving average of bit-rate is determined. 

5. The method of claim 1, wherein the content item is a commercial. 

6. The method of claim 1, wherein the digital video stream is MPEG 
20 compressed. 

7. The method of claim 1, wherein the content item is in an encrypted digital 
video stream. 

25 8. The method of claim 1, further comprising the steps of: 

obtaining broadcast schedule data indicating a beginning and/or end of 
broadcasting at least one content item, 

verifying whether said broadcast schedule data are in accordance with the 
detected boundary of a respective content item in the video stream. 
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9. The method of claim 1, further comprising a step (1 50) of determining a 

position of the detected boundary of die content item within a corresponding period of time. 

S 10. A device for detecting a boundary of a content item in a digital video stream, 

the device comprising: 

means (410) for determining an average bit-rate of the video stream over a 

period of time, 

means (420) for detecting a change of the average bit-rate indicating the 
10 boundary of the content item. 

11. A receiver for receiving at least one content item in a digital broadcast video 

stream, comprising the device as claimed in claim 10. 

15 12. A video recorder for recording at least one TV program, comprising: 

a receiver (430) for receiving at least one TV program in a digital video 

stream, 

the device as claimed in claim 10 in which the content item is the TV program, 

and 

20 - means (440) for recording the TV program based on its detected boundary in 

the video stream. 



13. A computer program product enabling a programmable device when executing 

said computer program product to function as the device as defined in claim 10. 
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ABSTRACT: 



The invention relates to a method of detecting a boundary of a content item in 
a digital video stream. The method comprises the steps of determining (130) an average bit- 
rate of the video stream over a period of time, and detecting (140) a change of the average 
bit-rate indicating the boundary of the content item. A moving average of the bit-rate may be 
determined. The method can be used for an encrypted digital video stream. The invention 
also relates to a device for detecting a boundary of a content item in a digital video stream. 
The invention further relates to a receiver for receiving at least one content item in a digital 
broadcast video stream, and a video recorder for recording at least one TV program. 

Fig. 1 
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